Skip to content

GH-37221: [C++] Refactor REE utilities with separating internal API, add flexible type matcher, and JSON support#48915

Open
HyukjinKwon wants to merge 1 commit intoapache:mainfrom
HyukjinKwon:GH-37221
Open

GH-37221: [C++] Refactor REE utilities with separating internal API, add flexible type matcher, and JSON support#48915
HyukjinKwon wants to merge 1 commit intoapache:mainfrom
HyukjinKwon:GH-37221

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Jan 21, 2026 •
edited
Loading

Rationale for this change

  • The C bridge tests (cpp/src/arrow/c/bridge_test.cc) required REE array encoding functionality for testing but had to depend on the ARROW_COMPUTE module (#ifdef ARROW_COMPUTE) to access compute::RunEndEncode(). This created an unnecessary coupling.
  • REE support into ArrayFromJSONString

What changes are included in this PR?

Before:

cpp/src/arrow/
├── compute/kernels/
│   ├── ree_util_internal.h        # REE encoding templates & helpers
│   ├── ree_util_internal.cc       # Allocation helpers  
│   └── vector_run_end_encode.cc   # RunEndEncode kernel implementation
├── util/
│   ├── ree_util.h                 # FindPhysical*, iterators
│   └── ree_util.cc                # Basic REE utilities
└── c/
    └── bridge_test.cc             # Calls compute::RunEndEncode()
                                   # #ifdef ARROW_COMPUTE

After:

cpp/src/arrow/
├── compute/
│   ├── kernel.h                   # Added match::REEValue declaration
│   ├── kernel.cc                  # Implemented REEValueMatcher
│   └── kernels/
│       └── vector_run_end_encode.cc  # Uses flexible matcher
├── util/
│   ├── ree_util.h                 # Public only
│   ├── ree_util.cc                # Uses internal macro
│   └── ree_util_internal.h        # macro + IsValueTypeSupported()
├── json/
│   ├── from_string.cc             # REE support added
│   └── from_string_test.cc        # Tests
└── c/
    └── bridge_test.cc             # Uses ArrayFromJSON directly

Are these changes tested?

Yes. All existing REE tests continue to pass, and added new JSON parsing tests for REE arrays.

Are there any user-facing changes?

Yes, this adds REE support as below:

#include "arrow/api.h"
#include "arrow/json/from_string.h"

int main() {
  auto ree_type = arrow::run_end_encoded(arrow::int32(), arrow::int8());
  auto result = arrow::json::ArrayFromJSONString(ree_type, "[1, 2, 2, 3, 3, 3, 4]");
}

@HyukjinKwon HyukjinKwon marked this pull request as draft January 21, 2026 01:08
@HyukjinKwon HyukjinKwon changed the title [C++] Extract the internals of the RunEndEncode kernel out of compute/ GH-37221: [C++] Extract the internals of the RunEndEncode kernel out of compute/ Jan 21, 2026
@github-actions github-actions bot added the awaiting review Awaiting review label Jan 21, 2026
@github-actions
Copy link

github-actions bot commented Jan 21, 2026

⚠️ GitHub issue #37221 has been automatically assigned in GitHub to PR creator.

@HyukjinKwon HyukjinKwon changed the title GH-37221: [C++] Extract the internals of the RunEndEncode kernel out of compute/ WIP: GH-37221: [C++] Extract the internals of the RunEndEncode kernel out of compute/ Jan 21, 2026
@HyukjinKwon HyukjinKwon force-pushed the GH-37221 branch 6 times, most recently from be96daf to 1faccbe Compare January 21, 2026 03:05
@HyukjinKwon HyukjinKwon marked this pull request as ready for review January 21, 2026 03:14
@HyukjinKwon HyukjinKwon changed the title WIP: GH-37221: [C++] Extract the internals of the RunEndEncode kernel out of compute/ GH-37221: [C++] Extract the internals of the RunEndEncode kernel out of compute/ Jan 21, 2026
@HyukjinKwon HyukjinKwon marked this pull request as draft January 21, 2026 03:23
@HyukjinKwon HyukjinKwon marked this pull request as ready for review January 21, 2026 03:38
@raulcd
Copy link
Member

raulcd commented Jan 21, 2026

@github-actions crossbow submit -g cpp

@github-actions
Copy link

github-actions bot commented Jan 21, 2026

Revision: 7e1f70b

Submitted crossbow builds: ursacomputing/crossbow @ actions-b9e0cacf84

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-debian-experimental-cpp-gcc-15 GitHub Actions
test-fedora-42-cpp GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-bundled-offline GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions

@raulcd raulcd requested a review from pitrou January 21, 2026 12:45
pitrou
pitrou reviewed Jan 26, 2026
View reviewed changes
@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Jan 26, 2026
@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Jan 29, 2026

The change is a bit large .. but a lot of them are moving the code around. Especially ree_util_internal.h.

@HyukjinKwon HyukjinKwon changed the title GH-37221: [C++] Extract the internals of the RunEndEncode kernel out of compute/ GH-37221: [C++] Refactor REE utilities with separating internal API, add flexible type matcher, and JSON support Jan 29, 2026
@HyukjinKwon HyukjinKwon force-pushed the GH-37221 branch 5 times, most recently from 3c362b5 to 7ac7f91 Compare January 29, 2026 22:30
@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Jan 29, 2026

Should be ready for a look.

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Feb 5, 2026

@github-actions crossbow submit -g cpp

@github-actions
Copy link

github-actions bot commented Feb 5, 2026

Revision: e1baf42

Submitted crossbow builds: ursacomputing/crossbow @ actions-b8a3e8cd32

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-debian-13-cpp-amd64 GitHub Actions
test-debian-13-cpp-i386 GitHub Actions
test-debian-experimental-cpp-gcc-15 GitHub Actions
test-fedora-42-cpp GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-bundled-offline GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitrou pitrou pitrou left review comments

Assignees

No one assigned

Labels

awaiting committer review Awaiting committer review Component: C++

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@HyukjinKwon @raulcd @pitrou